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Period for Reply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) OR THIRTY (30) DAYS, 
WHICHEVER IS LONGER, FROM THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 .136(8). (n no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 1 33). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent terni adjustment. See 37 CFR 1.704(b). 

Status 

1 )^ Responsive to communication(s) filed on 1 7 January 2006 , 
2a)S This action is FINAL. 2b)n This action is non-final. 

3) n Since this application is in condition for allowance except for fornnal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 1 1 , 453 O.G. 213. 

Disposition of Claims 

4) |EI Claim(s) 1-11 and 18-33 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) n Claim(s) is/are allowed. 

6) S Claim(s) 1-11 and 18-33 is/are rejected. 
?)□ Claim(s) is/are objected to. 

8) n Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) 0 The specification is objected to by the Examiner. 

10)0 The drawing(s) filed on is/are: a)n accepted or b)n objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1 .85(a). 

Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 
1 1 )□ The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-1 52. 

Priority under 35 U.S.C. § 119 

12)0 Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 1 19(a)-(d) or (f). 
a)n All b)n Some * c)^ None of: 

1 Certified copies of the priority documents have been received. 

2. n Certified copies of the priority documents have been received in Application No. . 

3. n Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 
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DETAILED ACTION 

Response to Arguments 

1. Applicant's arguments filed 1/17/06 have been fully considered but they are 
not persuasive. 

A. Applicant argues, "Schelling...do not discloses or suggest... 'accepting user 
input identifying a first concept of interest'... Schelling... do not disclose the recited 
'analyzing the mviltimedia information stored by the multimedia document to 
identify information relevant to the first concept of interest.'" (page 13, hnes 12-15). 

In response, Schelling is not relied upon for accepting user input to identify a 
concept of interest, rather for its teaching of generating a paper document for a 
multimedia document. Moreover, Choi discloses user input of keywords to identify 
a concept of interest, which has to be analyzed by the system to identify information 
that is relevant to the concept of interest of the user. 

B. Applicant argues, "Choi... do not teach or suggest 'accepting user input 
identifying a first concept of interest'... and then 'analyzing the multimedia 
information stored by the multimedia document to identify information relevant to 
the first concept of interest'..." (page 13, lines 16-22). 

In response, see response to argument A. 

C. Applicant argues, "Choi... is completely silent as to how the 'event' attributes 
in the free annotations are obtained, disclosing only that... information about who, 
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when, where, and what are described as its attributes in Event,. Since such 
information... is so very specific, at best, it seems that such information must be 
user provided. Regardless, no where in the reference do Choi... describe or suggest 
^ analyzing the multimedia information stored by the multimedia document to 
identify information relevant to the first concept of interest." (page 13, lines 23-29). 

In response, Claim 1 simply recites "accepting user input identifying a first 
concept of interest; analyzing the multimedia information stored by the multimedia 
document to identify information relevant to the first concept of interest..." 
Schelling discloses generating a paper document for a multimedia document and 
Choi discloses user input of kejrwords to identify a concept of interest, which must 
be analyzed by the system to identify stored information that is relevant to the 
concept of interest of the user. 

D. Applicant argues, "Schelling... merely teach printing the Index print'.. .the 
index print... does not include the recited annotations of the multimedia information 
relevant to a first concept of interest." (page 14, lines 6-10). 

See response to argument A. SchelUng teaches that the topic of interest 
(audio, video, text, etc.) is annotated using different styles. 

E. Applicant argues, "Lowitz...do not show Sprinting the multimedia 
information... wherein information that is identified to be relevant to the first 
concept of interest is annotated when printed on the one or more pages... Lowitz 
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...disclose selecting a portion of a frame of an input video data stream as printable 
image data and processing the selected portion of the frame to correlate the 
selected... frame of the input video data stream using at least one identifier... the 
identifier does not constitute the recited 'information that is identified to be 
relevant to the first concept of interest is annotated when printed on one the more 
pages.' Rather, the... identifier is merely a sequence number, and is not related to a 
user-provided 'first concept of interest.. .Nor.. .the. ..identifier constitute an 
annotation printed on paper that relates to Information that is identified to be 
relevant to the first concept of interest/ The identifier is simply sequence 
information.*' (page 14, line 11 -last line). 

In response. Schilling discloses generating a paper document for a 
multimedia document; Schelling teaches that the topic of interest (audio, video, 
text, etc.) is annotated using different styles; Choi discloses user input of keywords 
to identify a concept of interest, which must be analyzed by the system to identify 
stored information that is relevant to the concept of interest of the user; and Lowitz 
is relied upon its teaching of annotating relevant information when printed on a 
page. 



Claim Objections 

2. Claim 1 is objected to because of the following informalities: "when printed 
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on one the more pages" (line 9) should be changed to -when printed on the one or 
more pages-. Correction is required. 

Claim Rejections - 35 USC§ 103 

3. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 

obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as 
set forth in section 102 of this title, if the differences between the subject matter sought to be 
patented and the prior art are such that the subject matter as a whole would have been obvious at 
the time the invention was made to a person having ordinary skill in the art to which said subject 
matter pertains. Patentability shall not be negatived by the manner in which the invention was 
made. 

4. Claims 1, 4-7, 10, 11, 18, 21-24, 27-29, 32 and 33 are rejected under 35 U.S.C. 
103(a) as being unpatentable over Scbelling et al(US 6,706,097), Young-B Choiet 
a/ (article entitled ""An Integrated Data Model and a Queiy Language for Content- 
Based Retrieval of Video) and Lomtzetal (US 6,485,554). 

Claims 1, 11. 28. 29. 32. and 33: 

Schelling discloses a method for generating an index print (paper document) 
for a multimedia document storing multimedia information including video, audio, 
graphic, and text information. A user may select representations of the information 
for retrieving information of his interest. Schelling teaches identifiers (clues) that 
help the user to identify relevant information. Schelling teaches that users may 
print the multimedia information on a paper document (abstract; col. 1, lines 58*67; 
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col. 2, lines 1-14, 43-67; col. 3, lines 1-25; col. 4, lines 2-7). Schelling does not teach 
accepting user input to identify a concept of interest. However, Choi discloses a 
video data model that integrates feature-based model and annotation based model 
for improving content-based retrieval of video data that can be used in multimedia 
applications and which supports free annotations. Choi teaches the annotation- 
based model uses keywords to represent the video contents (abstract; page 192, first 
and second paragraph; page 197, par. 4.3; fig. 2). Therefore, it would have been 
obvious to one having ordinary skill in the art at the time the invention was made 
to modify SchelUng to include Choi's teaching of accepting user input of kesrwords to 
identify a concept of interest because it provides automated analysis, content-based 
video browsing, and an efficient way for users to retrieve specified video data. 
Schelling/Choi does not teach that information to be relevant to the concept of 
interest is annotated when printed on a page. However, Lowitz discloses a method 
and system for printing video images on a printable medium (abstract; col. 1, lines 
8-15, 35-38; col. 2, lines 6-34; col. 3, lines 23-60; col. 4, lines 8-12). Lowitz teaches 
that video information can encoded in barcode or the printable media (col. 2, lines 
35-49); a portion of a fi:ame of video data may be selected and stored as printable 
image data and processed to correlate the portion of the firame using an identifier; 
the identifier is a tag or mark associated with a specific frame and it can be printed 
together with the video data (col. 2, lines 50-62; col. 8, lines 23-64); and media data 
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(video, audio) can include annotations anywhere on the printable medium (coL 11, 
lines 44-67; col. 12, lines 1-20; col. 14, lines 57-67; col. 15, lines 1-14). Therefore, it 
would have been obvious to one ordinarily skilled in the art at the time the 
invention was made to modify Schelling/Choi's method of generating a paper 
document for mxiltimedia information including annotations because they provide 
the user with enhanced information that is easy and quick to assimilate. 
Claims 4 and 21: 

See claim 1. SchelUng teaches a method of generating an index print 
(printable representation) for a multimedia document having multimedia 
information (text, audio, video), (abstract; col. 1, lines 58-67; col. 2, lines 1-20). The 
method has layout information for printing the printable representation of the first 
and second tj^e on a paper medium (col. 3, lines 1-40). 
Claims 5 and 22: 

See claim 4. SchelUng teaches that the topic of interest (audio, video, text, 
etc.) is annotated using different styles (type indicators), (figs. 1, 4, 5). 
Claims 6 and 23: 

See claim 1. Schelling teaches an indicator icon that indicates files 
containing sound and a text message describing the data file. Shelling 
illustrates text relating to (fig. l) a sound recording of a person's (i.e. Grandma's) 
voice (col. 2, lines 62-67; col. 3, lines 10-29). Lowitz teaches indication of audio 
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information (col. 11, lines 62-67; col. 12, lines 1-7). 
Claims 7 and 24: 

See claim 6. Schelling teaches that the system analyzes the objects (text, 
image, etc.), retrieves and displays the information (fig. l; col. 2, lines 43-67; col. 3, 
lines 1-40; col. 4, lines 57-67; col. 5, lines 1-14). 
Claims 10 and 27: 

See claim 7. SchelHng teaches an index print having video fi^ames extracted 
from video content and text information extracted from audio content. 

Claim 18: 

See claim 1. SchelHng teaches a computer system having a processor, 
display, and memory (figs. 2 and 3; col. 3, lines 40-67; col. 4, Unes 1-7). 
Claim 30: 

See claim 1. SchelUng teaches an indicator icon that indicates files 
containing sound and a text message describing the data file. ShelUng illustrates 
text relating to (fig. l) a sound recording (audio information) of a person's (i.e. 
Grandma's) voice (col. 2, lines 62-67; col. 3, Hnes 10-29). 

Claim Bejectiona -35 USC§ 103 

5. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 
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(a) A patent may not be obtained though the invention is not identically disclosed or described as 
set forth in section 102 of this title, if the differences between the subject matter sought to be 
patented and the prior art are such that the subject matter as a whole would have been obvious at 
the time the invention was made to a person having ordinary skill in the art to which said subject 
matter pertains. Patentability shall not be negatived by the manner in which the invention was 
made. 

6. Claims 2, 3, 19, and 20 are rejected mider 35 U.S.C. 103(a) as being 
unpatentable over Scbelling/Cboi/Lowitzsjud. Niehen etaJ(US 6,056,642). 

Claims 2 and 19: 

See claim 1. Schelling does not teach that user input identifying a concept of 
interest is stored in a user profile. However, Nielsen discloses a system and method 
for allowing a user to create an interest profile for finding information of interest 
within a document. Nielsen explains that once the profile is created, it may be used 
on any document or web page to sort the information on the page according to the 
user's interests; once sorted, the information is displayed to the user (abstract; col. 
1, lines 59-67; col. 2, fines 10-22). Therefore, it would have been obvious to one 
having ordinary skill in the art at the time the invention was made to modify 
Schelling's system of retrieving and displaying information to include Nielsen's 
teaching of creating an interest profile because as Nielsen explains, it allows the 
user to view the points of interest without having to search the entire document. 
Claims 3 and 20: 

See claim 1. SchelUng teaches printing a matter descriptor such as a title 
identifying the subject matter of the data file (col. 5, fines 1-3). Schelhng does not 
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teach a relevance indicator for indicating a degree of relevance of the multimedia 
information. However, Nielsen teaches an interest profile that can include 
additional data such as relative importance (weights); points of interest are sorted 
according to relevancy and are presented into the document as Unks (abstract; coL 
1, lines 50-67; col. 2,Unes 1-9). Thus, it would have been obvious to one ordinarily 
skilled in the art at the time of invention to include Nielsen's teaching of indicating 
a degree of relevancy in Schelling's multimedia document because as Nielsen says, 
it allows the user to quickly spot the areas of interest within a document and move 
to a desired point of interest within the document or page (col. 4, lines 57-67; col. 5, 
lines 13-14). 

7. Claims 8, 9, 25, 26, and 31 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over ScheUing/Cboi/Lowitz2JiA. Gibbon etal iVS 6,098,082). 

Claims 8, 9, 25. and 26: 

See claim 1. Schelling does not teach that the mxdtimedia document includes 
printed closed-caption text information. However, Gibbon discloses a method for 
providing a compressed rendition of a video program in a format suitable for 
electronic searching and retrieval on the WWW. Gibbon teaches pictorial 
transcripts that are compact representations of video programs which are 
automatically generated by selecting representative frames or images from the 
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video program and combining them with a second media component such as audio 
or text which is associated with each representative frame (abstract; col. 1, lines 55- 
67; col. 2, hnes 1*15; col. 3, lines 10-15). Gibbon teaches that a printed rendition of 
closed-captioned text may be provided. The printed rendition is a pictorial 
transcript in which each representative frame is printed with a caption containing 
the portion of the closed-caption text corresponding to the scene from which the 
representative frame is taken (col. 3, lines 16-22). Thus, it would have been obvious 
to a person having ordinary skill in the art at the time of invention to modify 
SchiUing to include Gibbon's teaching of printing closed-caption text because it 
provides a printable visual presentation of the sound associated with the image 
(frame) of interest; therefore, close captioning is not only visible on a TV receiver 
designed to display it but it is also visible when being printed on paper. 
Claim 3i: 

See claim 8. Gibbon teaches that a printed rendition of closed-captioned text 
may be provided. The printed rendition is a pictorial transcript in which each 
representative frame is printed with a caption containing the portion of the closed- 
caption text corresponding to the scene from which the representative frame is 
taken (col. 3, lines 16-22). 
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Conclusion 

8. Applicant's amendment necessitated the new ground(s) of rejection presented 
in this Office action. Accordingly, TfflS ACTION IS MADE FINAL. See MPEP 

§ 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 
CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire 
THREE MONTHS from the mailing date of this action. In the event a first reply is 
filed within TWO MONTHS of the mailing date of this final action and the advisory 
action is not mailed until after the end of the THREE-MONTH shortened statutory 
period, then the shortened statutory period will expire on the date the advisory 
action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be 
calculated fi-om the mailing date of the advisory action. In no event, however, will 
the statutory period for reply expire later than SIX MONTHS from the date of this 
final action. 

9. Any inquiry concerning this communication or earlier communications fi-om 
the examiner should be directed to X. L. Bautista whose telephone number is (571) 
272-4132. The examiner can normally be reached on Monday-Thursday 8:00AM- 
6:00PM. 

If attempts to reach the examiner by telephone are unsuccessful, the 
examiner's supervisor, Weilun Lo can be reached on (571) 272-4847. The fax phone 
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number for the organization where this application or proceeding is assigned is 571- 



10. Information regarding the status of an apphcation may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR 
only. For more information about the PAIR system, see http://pair-direct.u8pto.gov. 
Should you have questions on access to the Private PAIR system, contact the 
Electronic Business Center (EBC) at 866-217-9197 (toll-free). /\ 



273-8300. 
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